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(57) Abstract 

A method and apparatus for monitoring a region of a body surface. The apparatus includes means for recording (160) first 
and subsequent multispectral digital images of the surface including the monitored region. First and subsequent images are high 
magnification images which can be recorded at low magnification. Diagnosticalty useful classifications (324) of pigmentes skin 
lesions are formed using a database (326) containing quantitatively extracted features from images recorded for a plurality of 
skin lesions, and correlating the features from each such lesion in the database with the medical history of the skin lesion from 
which the image was recorded. A diagnosis of premelanomatous or early melanomatous condition can be made by using the ap- 
paratus to characterize a surface region including a skin lesion and comparing lesion features with features of lesions previously 
stored in the database. 

See back of page 
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Apparatus and method for imaging and classifying lesions 

Background of the Invention 

5 This invention relates to an improved apparatus and 

method for in vivo monitoring of surfaces of the body 
Surfaces, such as the skin are directly accessible, while 
other surfaces, such as the cervix, the retina, and the 
vascular endothelium require an optical apparatus to be 
10 visibly accessible. 

An example of a surface commonly monitored for - 
abnormalities is the skin. Skin cancer incidence has 
increased markedly in recent years. Present clinical 
methods of screening and monitoring cutaneous surfaces for 
premalignant features include, qualitative assessment of 
gross v 1S ual features (such as asymmetry, border 
irregularity, color variability, diameter and elevation of 
skm lesions, , and sequential examination, relying on the 
20 f 7 '"" 0 '' mem °^' ascriptions, sketches, and 

visual inspection of conventional analog photographs, to 
assist n deciding if a surface lesion is new or has 
features which have changed. Patients with the dysplastic 
nevus syndrome may have hundreds of p lgiMat . d Bk J Lions 
25 ^ C V eed t0 be stored for early signs of cancer or 

premalxgnancy. A system which systematically a > monitors 
the number of iesions and their positions on the body 
surface, and/or (2, quantifies morphologic and spectral 
features of such lesion ions would provide a useful tool for 
improving management of this Potentially deadlydisease . 

is the- T SUrfaCS - r ° Utinel r -nitored for abnormalities 

is t he c cancer- of the cervix is a common cancer in 

women. Digital colposcopy with simnlo < 

been S hn„n „ simple image enhancement has 

been shown, anecdotally, to detect premalignant cervical 
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abnormalities in cases where the standard screening test 
the Papanicolaou smear, was falsely negative 

i. ...^"L M T" " 3 SUrf3Ce WhSre ViSUal —^tlon 
5 the ! ° ^ abn °^ aiit -3 or changes is 

deVe^T"' Retinal imaging ±S imP ° rtant -thoc. for 
detecting, monitoring, and guiding therapy for clinical 

metholT t0 blind — ^togrammetric 

methods for obtaining three-dimensional measurements of 

10 dig^l m'etTdT ^ anal ° g Ph0t0 ^ a P^ «- simple 

digital methods for two-dimensional imaging of retinal 

features are presently in the vanguard of methods used to 
monitor certain retinal features. 

The vascular endothelium, however, is the anatomic 
surface responsible for the majority of deaths in this 
» country (i .e. due to myocardial infarction) . 

method I «»".copic catheters, yet no systematic 

method for analysing and utilizing images of this surface 
previously has Jbeen developed. surtace 

20 surface!" ViSU3lly anatomic 

letW ; 3 SynthSSis ° f -PP-priate digital 

methods for systems identification and computer vision would 
-Prove the detection and monitoring of abnormal!:;:: 
2t b6fore ' d ^ing, and after therapy. ' 

Stannary of thm Invention 
I* la therefore an object of this invention to 
provxde an apparatus and method for (l) the In vivo 
detection qualification and/or classification of features 
of biological surface structures and processes ,21 T 
30 monitoring of changes in such feature! wi^ tlle or L 

such features, and/or ,4) the reconstruction of 3- and 
^-dimensional (3-D as a function of time) surface Z 

35 acf^r ^^^^^ - d ^ogy of SUC h visually 
35 accessible anatomic surfaces. • 
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These objects of the invention are attained by 
combining methods derived from systems identification and 
computer vision with methods of photography that are 
(1) digital, (2) quantitative, (3) multispectral , 
5 (4) multiview, and/or (5) multimagnif ication . Digital 

photographic methods provide a computer-based substrate for 
automatic feature extraction, surface reconstruction, tissue 
characterizations and feature location mapping. 
Quantitative digital photography provides the ability to 

10 precisely and reproducibly estimate quantitative features of 
anatomic surface structures or processes. Multispectral 
digital photography provides the ability to characterize and 
classify surface structure components and processes, and 
their temporal-spatial distributions, in particular, the 

15 depth of their subsurface extents. Multiview digital 
photography (or single-view photography with 

multiple-position or structured-light stimulation or single- 
view photography with a single light source) provides the 
ability to reconstruct three-dimensional surface elevator 

20 and topology using, for example, stereo, optical flow, 
photometric stereo, shape-f rom-shading, moir&, or 
structured-light methods of three-dimensional surface 
reconstruction. Multimagnif ication or multiresolution 
digital photography provides the ability to map the global 

25 surface location of macro- or microscopic surface features. 

More specifically, it is an object of this invention 
to provide an apparatus and method for the acquisition of 
computer-based digital photographs of skin lesions, 
permitting, not only efficient storage and recall of visual 

30 documentation, but also automatic counting of lesions and 
mapping of their global body surface locations, and 
quantitative analysis of visual features thereof, as a means 
for detecting evidence of premalignancy or malignancy. 
Further, by means of quantitative comparison with previous 

35 images, and the extracted features thereof, the detection of 



SUSSTiTii IE SHEET 



Printed from Mimosa 01/01/11 10:27:34 Page: 5 



WO 90/13091 



PCT/US90/02015 



- A - 



20 



25 



30 



new skin lesions and changes in morphologic and colorimetric 
spectral, features of existing lesions may be achieved 

Moreover, it is yet another object of this invention 
to prov.de an apparatus and method for the simultaneous 
digital acquisition of color and infrared photographs of 
skin, permitting the implementation of a novel quantitative 
algorithm for estimating the depth and volume of certain 
cutaneous structures or processes. Such estimates are 

> leptrof 7 - 11 " 1503311316 Wlth r ° Utine anal ° g ^ 0t °*«P*y. Since 
dept h of invasion „ one of the most important prognostic 

features of melanoma, such a method for imacing below the 
surface of the skin would provide important clinical' 
^formation that cannot be obtained with present methods of 
noninvasive clinical examination. m addition, other types 
of spectral algorithms may be applied to such multispectral 
images to characterize the spatial distribution of the 
various light absorbing components of skin, including 
pigments, hemoglobin, and bilirubin [1]. 

Similarly, it is an object of this invention to 
provide an apparatus and method which, when combined with 
approp riate c i inical examination equipmentf may prov . de 
ability to quantify features of multispectral digital 
Photographs of the cervix in an effort to enhance the 
diagnostic accuracy of colposcopic examination. m 
addition, it is yet another object of this invention to 
provide an apparatus and method which may be applied to 
quantitative, automatic three-dimensional retinal imaging, " 
with quantification and classification of anatomic an! 
Physiologic features thereof, thus providing improved 
methods' to detect and monitor retinal pathology B y 
quantifying 3 -dimensional features of the topology of 
retinal surfaces and the retinal vasculature, such methods 
»ay provide the means for improved management of retinal 
detachment, glaucoma, and diabetic retinopathy 
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to D M ° r£OVer ' ifc is ^ - ^ther object of the invention 
to provide an apparatus and methods which when combined with 
an apparatus that provides visual accessibility, would 
permit in vivo multispectral inspection and tissue 
characterization of the coronary artery endothelium in the 
region of atheromatous plaque. Such may be accomplished by 
-aging reflected li g ht of one or more wavelength" passed 
through an angioscopic catheter, followed by multispectral 

10 Zr^ 1 ^ 5 tiSSUS Parameter ^«i«tion. Such a 

10 technique could provide a novel method for precisely 

oTo^er^ - ™«Y artery lesions or 

of other lesions in other vascular structures 

and/o Th " S . qUailtitati -' multispectral, multiview 

and/or multimagnif ication photography of anatomic surface" 

ci;:i:; ^.^-^ *™ ..tw±o» and/or tis c ; u s ; 

c ra= terizatlon may provide importa ^ advan 

"IT 611 ' ° f 3 ° f diS6aSeS " -chnology, L 

"I :: n ; 0 a ; h coramerciai n 0n ^ ca r: rBas 

20 eye color or k qUMtitatiVS °* ^ and ■ 

III or skm texture and dryness, for use in the 

choice or testing of cosmetics and pharmaceuticals. 

Bri.f D««cxiption of th« Drawing, 
The drawing is briefly described as follows- 

25 ha ,H Flg " 1 " 3 bl ° Ck dia ^ ram of embodiment of the 
25 hardware of the invention; 

the inJe" 5 ; 2 A ! " bl ° Ck dlagram ° f Sample e ^o^»ents of 

D«««rIption of Pr.f.rr.a-tobodiawt. 

div^H / 3ySt6m m ° nit0rin 5 surfaces of the body is 

divided naturally i„ to two parts. The first part L 

of the hardware of fc St part consists 

35 software in ^ ^ ""^ C ° nsist * the 

6 ° r Pr ° gramS WhiCh ««- the hardware to function as 
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desired. This software however may be implemented directly 
in hardware. u " 
HARD WAKE 

The hardware for . system (or monitoring visualiv 

centre! processing unit (CPU) in is connected by one or 
»ore buses 10e to the following apparatus: a real only 
memory ,rom, 116, a random access memory (RAM) lis one or 
»ore digital input/output ports 110 to which r . attached 
« =»e or more input devices such as , Keyboard 11 2 and a 

de : iCe , 111 ' a stroller i 2 . to which is 

contf ^ * P ^ menitC ' 1 13 °' °" e « '""9. device 

£ -teeing addition.! hardware^ \'' t ^ 
imagmg subsystem 133 capable of at least 24 „, „ ! 

i»ege acquisition, processing, and display a^d COl ° r 

ODf ^ — . _. _ ^ _ a ' tU5 P ia y^ and one or more 

20 *^LT""™ 120 <° computation speed. 

or mor S St ° re " i " a9l " g sub =^««> "3 includes one 

138 for storrng image,, da ta. and/or microcode, a^raphics 

m^rT , T —"on interface 148 . The coprocessor 1 2 0 

«J 140 tTf " — -th «h. graphics 
afld the frame store RAM 138. 

The image input port 136 is connected to on* 

-■ ==r= = =: — £f 

.digital.toput-ana/or-output -devices. For e,a„pl e L „ 

co to , input/output intetJa ^ *• -T * 

reading the status of on. or more buttons or switches (222 

3= it'may be conTec'LTt" ^ ^ '™ C " "0. and 

ey be. connected to an optional position sensor 
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controller ux.it 152 which converts" information from a 
position sensing element (224 Fig. 2(a)) contained in the 
image transducer device 160. 

The digital input/output ports 110 and 134 and the 
bus interfaces 132 and 148 provide the flexibility to add 
additional devices such as a speech input/output device 131 
for the convenience of the operator or to provide verbal 
instructions to the operator and/or patient. 

The. image input port 136 contains a set of 
10 analog-to-digital converters (A/D) , for example four 

Parallel A/D's, for use with an analog image transducer 

eV1C6 16 °- Alterna t±Vel * At -y include appropriate 

digital input hardware for use with a digital image 
transducer device 160. Each A/D in the image input port 136 

wi:: ert ; h an . anaiog s±9nai from an ^ ~ 

withan the image" transducer device 160 to digital values for 
storage xn the RAM 138 of the frame store or imaging 
subsystem 133. The memory 138 stores one or more digital 

lT,ii\:ir?~^ size <up to at ieast 1024 « io2 <> 

Pxx.ld.pth (up to at least 32 bits) . The data may be 
rap.dly converted to other digital values before storage in 

"° * »" " •* * « i-P« lookup-tables (a^ 

contaxned „ the image input port 136) . After storage it 
-ay be mathematically manipulated by the graphics CPU 140 
master CPU 114, and/or additional coprocessors 120. The ' 
data „ then read (via output looJc-up-tables) by a set of 
digital-to-analog converters (D/A's) contained in the image 
output port 142 and displayed on a 2-dimensional or 
3-dimensional stereo, video display monitor 144. The output 
Port 142 may contain additional hardware for drivina other 
types of displays, "for example a 3-dimensional holographic 
display. The image output port 142 is also connected to a 
color hardcopy printer 14 6. 

Imag* tranaducar device: 



20 
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The image transducer device 160 may contain one or 
more cameras or other imaging devices which may be directed 
to view a visibly accessible anatomic surface of a patient 
or subject, either through one or more lenses with 
5 appropriate beam splitters, filters, filter wheels, or 
shutters, if necessary, or through an examination 
microscope, colposcope, or ophthalmoscope 260 (Fig 2(c)) or 
angioscope 250 (Fig. 2(c)) with appropriate image sensors 
251, or by means or other optical or electro-optical 
10 apparatus. The image sensors contained in the image 
transducer device may be monochromatic, colors and/or 
infrared, and may consist of one or more charge-coupled 
devices (CCD) , vidicon tubes, confocal laser scanners, or 
other' type of image sensor with appropriate filters where 
IS necessary. An R-G-B color camera ideally should have precise 
registration between the reds green, and blue imaging 
sensors . 

A preferred embodiment of the image transducer device 
160 is illustrated in Fig. 2(a) and Fig. 2(b). m this 
20 embodiment the device is divided into a global surface image 
transducer device (Fig. 2(b)) and a local surface image 
transducer device (Fig. 2(a)). The global surface image 
transducer device (Fig. 2(b)) is designed for the purpose of 
obtaining images with a field of view encompassing a 
!5 relatively large, i.e. .global, portion of the visible 

anatomic surface under examination. Global images may be 
used as maps onto which the location of higher magnification 
"nages, obtained with the local surface image transducer, 
may be recorded either manually by the operator or by use of 
an input device (e.g. a pointing device 111), or 
automatically by means of an -automatic position sensor 224 
246, and 152. In addition, global images may be used 
independently of local images to characterize larger body 
surfaces up to and including the entire surface of a body 
For example, global images can be used to count analyze, and 



SUBSUME SHEET 



Printed from Mimosa 01/01/11 lfV?fl 05 Pan*- m 



PCT/US90/02015 



10 



15 



- 9 - 

temporally monitor all lesions on a body. The image input 
port, 136 may be configured to switch, under software or 
operator controls between the global and local surface image 
transducer devices; or if necessary, a video switching 
devices controlled via the digital input/output port 134, 
may be interposed between the image transducer devices 160 
and the input ports 136. 

Global imaga transducer: 

The. global image transducer device (Fig. 2(b)) 
consists of one or more video or digital cameras or other 
image sensing devices 242 with appropriate lenses such as 
wide-angle or remote controlled zoom lenses 24 6 and ' 
illumination sources 244 which illuminate a large or 
"global" region of the anatomic surface with uniform light 
structured light, or light with an arbitrary spectral 
distribution specified by a broad or narrow bandwidth. The 
apparatus may be mounted on a stand 240 over a patient 
examination table 249 or alternatively on a 
computer-controlled robot positioning arm 240 [2] The 
stand or robot arm 24 0 may also be configured to direct the 
camera ( S , (or other image sensor (s) ) 242 to view a standing 
or sitting, rather than a supine, patient or subject. The 
source element for an automatic position sensing device 248 
may be mounted on or near the examination table 249 or for a 
standing patient, anywhere near the location where the 
patient or subject stands during global surface examination, 
other image or energy sensors may be mounted in a specified 
configuration to automatically determine the position of a 
local image transducer, e.g., in a machine vision method to 
detect the location of a local image transducer. 
• X»ocal ioag* truaduetr; . 

The local surface image transducer device (Fig 2(a)) 
is designed to obtain a magnified view of a relatively 

35 IT* 11 ^ l0Ca1 ' fleld ° f ViSW ° f thS anat ° miG 

228 under examination. The local surface image transducer 
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.is designed to simultaneously transduce true color and 
infrared (or other multi 3 pectral set of) images of reflected 
and/or emitted light from the anatomic surface under 
examination. An R-G-B video or digital camera 210 (e g 
5 with three CCD sensors) and an infrared video or digital 

camera 212 (e.g. with the same type of CCD chip as the r- G -b 
camera) are both attached via a beamsplitter 214, with 
appropriate optics for matching the optical distance between 
the lene and the imaging plane of each device to a 
10 macrolens 216, which is supported by a positioning cone 220 
made of non-metallic or metallic material. The positioning 
cone 220, which may rest directly on an accessible anatomic 
surface 228, acts to simplify and S peed image acquisition by 
proving a f i*ed working distance between the imaging 
15 deva.ce and the anatomic surface. 

Illumination may be provided by an annular fiberoptic 
light guide 218, mounted between the macrolens 216 and the 
positioning cone 220, such that white, colored 
monochromatic, coherent, or structured light is directed 
uniformly within the positioning cone. The type of light 

TZTTt tbr ° U9h fiber ° PtiC *«±d. is controlled 

by the .light source 154 (Fig. 1} wnich may contain ^ 

light sources and filters under electronic control by the 
25 inpUt/ ° Ut P Ut -terface 150 or port 134. However, a 

25 ^ngle white light source may be used to obtain three or 
four simultaneous- images - red, green, blue, and/or 
infrared - in tals embodiment, when three or four parallel 
analog- to-digital converters are contained in the image 

-put controller 136 of F igl . If an ultraviolet source is 
30 -1 ded in the light eource module ^ ^^^^ in -s 

the local- surface- ima^ transducer, then a sequential image 
of ultraviolet-stimulated fluorescence of the anatomic 

ZTnllZ* ^ °* tained - ThS "^nation may be polarized 
35 ^ " ClUdin * a Polarization device anywhere between the 

light source 154 and the positioning cone 220, for example 
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-li- 
as a flat, round polarization filter between the annular 
fiberoptic light guide and the positioning cone 220. 

The local surface image transducer Fig. 2(a) is 
designed to be hand held and contains one or more 
tactile-feel pushbuttons (or other type of switch) 222 which 
are used by the operator to control the hardware and 
software, for example to signal the apparatus of the exact 
instant to acquire one or more images from the image 
transducer. apparatus, to switch between the local and global 
transducers, or to calibrate the position sensor device 152. 

The sensing element 224 of the position sensor device 
152 is mounted at a fixed location on the local surface 
image transducer device, whereas the source element 248 is 
mounted at some reference position, for example on or near 
15 the examination table 249. Thus the location of a high 

magnification image obtained by the local transducer may be 
mapped onto a low magnification image obtained by the global 
transducer. Various position sensing hardware may be used 
including but not limited to magnetic field generator and 
sensor and machine vision camera with or without special 
source lights, such as light emitting diodes. 

A small, removable calibration structure or chart, of 
known shape, size, and or color (s), may be placed at the' 
distal opening of the positioning cone 220 during the 
acquisition of an image for the purpose of calibrating for 
spatial, intensity, and spectral measures. 

Note. that there are a number of alternative 
configurations for these image transducing devices. For 
example, the positioning cone 220 and annular fiberoptic 
light guide may be replaced by an integrating sphere and a 
set of .fiberoptic bundles respectively. - The macrolens 216 
may be replaced by a zoom lens or by a stereo macrolens 
pair, and the infrared camera 212 may be replaced by another 
R-G-B camera to obtain stereo color images for image surface 
35 reconstruction. Alternatively, two R-G-B cameras -210 and 
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two infrared cameras 211 may be connected via an additional 
beamsplitter 214 to a stereo macrolens system 216 for use in 
both stereo surface and stereo subsurface reconstruction 
Moreover, multispectral images may alternatively be obtained 
by substituting a single camera with a filter wheel (under 
digital control via the digital input/output port 134) for 
the camera pair 210 and 211 and the beamsplitter 214, 
respectively . 

Additional Imago Acquisition Appar-atus 

A "local" image, as that term is used herein, is a 
hxgh magnification image, and a "global" image, as that term 
xs used herein, is a low magnification image, as defined 
above. Generally, a local image presents a magnified image 
of, for example, a pigmented skin lesion that is of 
particular interest; in this example, the global image might 
be a low resolution, low magnf iciation image including the 
skxn lesion and also including enough of the surrounding 
body surface so that the location of the lesion can be 
unambiguously indentified. A global image might, for 
instance, show the upper right dorsal quandrant of a person 
navxng a mole, located over the right shoulder blade, while 
th, mole itself might nearly fill the corresponding local 
linage frame - 

A "high magnification" image, as that term is used 
*rexn, is an image having sufficient resolution to permit" 
quantitative extraction of selected features, or display of 
certain attributes from- the image, as described herein A 

low magnification" image as compared to a high 
magnification image, is an image that can include the 
regxon of surface contained in a high magnification image, 
and generally includes a greater area of the body surface; 
the low magnification image may be principally employed for 
locatx^g the position of positions of one or more high 
mangification images, and need not have sufficient magnitude 
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or resolution to permit quantitative extraction of selected 
features . 

A "local" image, as that term is used herein, is a 
high magnification image, and the image frame is l argely 
5 occupied by a particular structure or particular nearly 
adjacent structure on or in the suface being imaged. A 
"global" image is an image whose frame includes a 
substantial area of the surface being imaged, such as, f or 
example, an aspect of an entire libm, or of the upper right 
10 dorsal quandrant, or of the right side of the face A 

global image can be either a high resolution image, or a low 
resolution image, according to its use. As described in 
more detail herein, a global image can be used in connection 
wxth one or more local images; or a global image or a time 
series of global images can be used without reference to any 
local images, as- for example to identify and/or to monitor a 
population of skin lesions, which may be changing over time. 

The present invention employs methods for affecting 
the quality of skin images by reducing index of refraction 
20 mismatches and/or controlling the polarization of the 
incident and the reflected light as seen by the image 
transducer. 

One method, epiluminescence microscopy [3] i s £ 
standard technique employed by dermatologists in which one 
Places several drops of oil on the lesion and then presses a 
glass slide over the oil-covered lesion. This set-up is 
observed under a microscope. The purpose of this method is 
two-fold: l) Because the index of refraction of oil is 
closer to skin than air, the effect of multiple index of 
refraction mismatches is reduced. 2) By pressina on the 

10n ' the 3-dimensional surface structure of the oil is 
flattened out, virtually eliminating specularity (as long as 
the camera is not exactly on axis with the illumination) . 
The two major disadvantages of oil immersion microscopy is 
that it is messy to work with and cannot always be applied 
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to lesxons in inconvenient body surface locations or lesions 
which are significantly elevated above the surface of the 
I"' M ° re im P^-tl y , though, the compression distorts 
the l.axon, and causes capillary blanching, which may 
distort pigment patterns and may make it difficult to view 
capillary patterns. 

Another method involves placing a polarizer in the 
- Path of the light source to polarize the incident light 

10 P ° lari2er " PlaCed " «»• Path of the reflected 

Ught to allow emphasis of certain components of the 
relected light to reach the image transducer. When the 
Polarizers are parallel to one another, the image observed 
by the camera emphasizes the specular reflectance off of the 

15 to el" f^™*" ^ ™ Perpendicular 

to one another the reflection off of the surface is 

ruch im a S 2e th al T ing ° bSerVati0n ° f — P~ structure 
such as : the pigment and capillary patterns. if the 

Perpendicular Lnage is subtracted from the parallel image, 

20 Tol Prlmarily SPeCUlar ~^ctance, which 

contaans infection of the surface structure of the skin 

ttph . ^ P reser * invention employs two variations on the 
technique of epiluminescence microscopy. m the first 

gILT r tta * S ^ ent »«*««• is used to replace 

5 « U a r b ?T Sin5 th6 —P-ssion of the lesion and 
: capillary blanching. This membrance may be mounted on a 
snap-on attachment to the local image transducer or 
positioning cone described above. m the second variation 
a small unobtrusive amount of oil is applied to the skin 

> rigT:rfir t iarly poiari2ed (with - ^ " 

con£ " US6d ln fr ° nt ° f imag6 *«n,duc.r. The 

^ lter the effect" 

of refractive index mismatches without compression of the 
lesion or mess. By acquiring images in this way, the 
quality of the image obtained in epiluminescence microscopy 
is improved. Purther, by employing the quantitative feature 
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extraction methods of the present invention, the technique 
of epiluminescence microscopy can be made quantitative and 
objective . 

It will be apparent to those skilled in the art that 
many combinations of oil and polarizers can be used during 
image acquisition to emphasize a desired characteristic of a 
skin surface. For example, specular reflectance can be 
better imaged by subtracting an image obtained using oil 
from the parallel image described above. Each of these 
combinations can be used in the image acquisition protocols 
of the present invention including the various lighting 
conditions described herein and can be used in either a 
local or global imager. 

The term "moistener", as used herein, means a 
substance that descreases effects of refractive index 
mismatches, as caused, for example, by single or multiple 
air-skin interfaces; moisteners include, for example, water 
or an oil such as mineral oil. 
Lighting hardware: 

Lighting conditions may be uniform or structured with 
respect to space and/or the electromagnetic spectrum. For 
example, spatially uniform white light may be provided by a 
white light source connected to an annular fiberoptic 
ringlight or integrating sphere. Spatially structured 
light, such as the stripes, random dot patterns, or moire 
patterns used with certain three-dimensional surface 
reconstruction algorithms, may be provided by a white light 
source with an appropriate set of filters and/or slides or 
by a laser with appropriate electro-optical apparatus. 
Spectrally structured light may be provided by an 
appropriate set *6f filters with a white light source, or by 
one or more lasers (e.g. a laser that emits ultraviolet 
light, which may be used to stimulate melanin fluorescence) . 
Lighting conditions may be standardized to a set of spectral 
and spatial distributions (e.g. spatially uniform white 
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light of a known color temperature and intensity which 
contains a fcnown distribution in the near infrared region) 
Polarization devices may be used, in addition to minimize ' 
specular reflections. 
5 The term "illumination", as used herein, means 

electromagnetic energy, including for example and without 

* t " t4M Snergy ±n the -^ible UV, . and ir wavelength 
regions . 

10 is an A " mUltiSpeCtral " ™***> « "at term is used herein, 
3 311 image co «^ructed from a plurality of spectrally 
Afferent images, which can be made concurrently or 
nonconcurrently, and which can be made spectrally different 
*y manipulation either (or both, of the radiation used to 
15 surfTrr th6 h SUrfaCe ° r ° f thS "^tlon received from the 

This 1 T ima9S ° r ±mageS a " bein * «cord.d. 

This may also delude manipulation of the images after they 
have been recorded. 7 

Wcteo-optie.1 iniag. guid.., iKitch8rs . 

20 and from"! 563 ^ °™ °* "° r - de -«s 

and from one or more spatial positions simultaneously, or 
sequentially, to permit, for example, multispectral 
acquisition (e.g. r- G _ b co1o „ infrared and 

ph!"Ie t ° let ' StimUlated flUO "— > or Photogram.etric or 
25 ITZZ * ri !. Stere0 ' ° r ° Ptical three-dimensional image 

^construction. An optical device may be used to permit Le 
acquisition of multispectral or multiple view images. L 
multispectral imaging from one views one or more Lam 
ZllTr, Wlth aPPr ° Priate Properties or other 

30 tTtl 0 r ^ ^ US6d ^ ^ *«» ~. view 

to two or more image sensors, or alternatively one or more 
filters may be automatically switched in and out of tlT 
-age path, either mechanically or electro-optically, to 

3^ - ^ired, for e^e for = Z^^^T" 
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from two or more lenses may be directed sequentially to a ' 
single set of image sensors by means of an electro-optical 
or mechano-optical switch. Alternatively, an electronic 
switch may be used to rapidly switch between image sensors 
5 which view the same object from a different spatial position 
or with a different spectral sensitivity. A calibration 
ruler, grid, chart or other structure may be imaged and used 
for automatic calibration for spatial measures, gamma 
correction, and color. 
10 Position itnaor: 

The three-dimensional coordinates of the field of 
view of the image transducer with respect to the patient are 
sensed by a three-dimensional position sensor which is 
mounted on or near the image transducer. The output of the 
position sensor is input to the computer via an input-output 
port 134, or 110 or bus interface 148 or 132. 
SOFTWARE 

Fig. 3 is a block diagram of the system functions and 
information flow in the system described below. These 
functions may be implemented in software, firmware, or 
hardware. The discussion below is for an embodiment of 
these functions in software. 
Initialisation : 

Upon system boot the master CPU 114 (Fig. 1) executes 
the system programs stored in the system ROM 116 and 
initializes the system. The programs for monitoring 
anatomic surfaces are read from magnetic disk 124 into RAM 
118 and 138 and are executed by the master CPU 114, graphics 
CPU 140, and additional coprocessors 120 if installed. 
These programs specify a set of default routines, protocols, 
algorithms, parameters, and hardware devices which are to be 
employed during image acquisition 312 (Fig. 3) analysis 
314,316,318,328,322,324, storage 326, display 328, and if 

appropriate, computer-aided decision, management, or control 

35 332. 
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Setup: 

e » «. eXeCUti ° n ° f the SUrface monitoring program, a 

control display screen may be displayed on a video monitor 
144 and/or 1 16 . Using one of the several input devices 
5 111,112,222, or 131, the operator 330 bay execute a setup 
routine which allows one to change the default routines 
protocols, algorithms, and parameters which were set during 
initialization above and which are described below The 

10 ZiT t0r 3 ' ° ^ Ch ° OSeS eXample ' 3 -Magnification, 

10 multiview, multispectral and/or multi-illumination 

acquisition protocol. These and other choices will 
determine the specific hardware (Figs. 1, 2 , , system' 
functions (Fig. 3), and algorithms that will be employed 
15 TlTJ 5 ,T analySlS 314 ' 316 ' 318 '328,322,324, storage 326, 

le fT, ^ ° Perat - 330 ^ <=hoose to turn on or 

the following optional system functions — surface 

IlT S r^ iOD 318 ' ClaSSifiCati - 324, and decision and 
control 332. xhe operator 330 may also choose whether the 

20 IL , fUnCti ° nS W±11 Pr ° Ceed automatically, with only 
minimal operator input, or manually with additional 
opportunity for the operator to change or choose the 
specific routines and parameters employed by the functions 
as tney proceed. 

Imag« Acqui.ai.tion (312) : 

uoon ^ A ° qUi " iti0n P«oool. Referring to Figs. 1 and 3, 
npon the command of the operator 330 one or more images as 
determined by the initiali.ation and setup routines, of L 
region of interest on the anatomic surface (e.g. 228) is 

-LTlocatel ^ ^ entSred thr ° Ugh 3 P-^tton 

switch located on or near the image transducer device 222 

.the keyboard 112 or pointing device 111, or for added ' 
convenience may be issued verbally and interpreted by the 
speech processing unit 131. 

The command to capture an image results in the 
storage of one or more frames of data (as determined by the 
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initialization defaults and the setup choices) in the frame 
store RAM 138. Coincident with the acquisition of an image, 
the 3-dimensional spatial coordinates of the imaging 
transducer 160 may be sent via the position sensor 
5 controller 152 to one or more of the CPU's 140, 114, or 120 
for further processing and subsequent position mapping on a 
global reference image or synthesized surface map. This 
acquisition sequence may be repeated to acquire an arbitrary 
number of images from different locations on the surface 
10 Numerous variations of this acquisition protocol may be 
chosen by the operator 330 during the setup routine 
described above. 

Automatic *.p«ct ratio correction. As determined by 
the choice of acquisition hardware and software in the 
initialization and setup routines above, software commands 
which control the A/D converters in the image input port 136 
and the graphics CPU 140 are used to control the input 
aspect ratio of the captured video image. For example, even 
though the aspect ratio of a standard NTSC video signal is • 
20 4:3, it may be captured with a 1:1 aspect ratio by adjusting 
horizontal and vertical zoom commands which control the 
factor by which a pixel clock signal is divided before it is 
used to drive the A/D converters. The ability to convert to 
a 1:1 aspect ratio during the analog-to-digital conversion 
25 process simplifies the computations involved in subsequent 
quantitative feature extraction and permits the use of a 
high resolution display screen with a 1:1 aspect ratio where 
multiple images and graphics materials may be simultaneously 
displayed. This variable input zoom may also be used to 
30 rapidly switch between cameras with different types of video 
signals-as specified by tire acquisition protocol chosen in 
the setup routine above. 

Multiaagnificmtion acquisition. if so determined 
during the initialization and setup routines, under the 
35 control of one or more input control devices 
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222,111,112,131, images are acquired at two or more 
magnifications. This may be accomplished by using two 
different image transducers or a single image transducer 
vith changes in the optical or electro-optical device which 
focuses the image onto the image transducer. Fcr example 
the first set of images may be acquired from an image 
transducer device which has a field of view encompassing a 
large portion of the visibly accessible surface (i e low 
magnification), for example by a camera mounted above a 
supine patient, or at a fixed distance from a standing 
Patient (Fig. 2 (b) , . This first set of images should 
encompass most of surface at low magnification in a small 

of the global positions of subsequent high-magnification 
images which have smaller fields of view, with respect to 
the overall anatomic surface, or region thereof. Th - 
position of the high-magnification images are manually ( e g 
via an input device 222,lli, 112 , or 131, or automatically 

20 il'l^T^ T POSiti ° n SenS ° r 152 ' 224 ' and 24 *>- -corded 
20 on the large field of view images. 

in addition, multiresolution images, each with the 
same f ie id of view, may be acquired in rapid sequence, using 

single image transducer with a single setting of the 
optical apparatus, simply by changing the horizontal and 
vertical pixel clock divide, or zoom, factors of the 
analog-to-digital converters in the image input module 136. 

the TV* " Sn,a11 3126 ±mage ' " - ^ « 

fuller SCrSen " 4 ' ^ ^ a<=qUired i—aiately after a 
full Size by increasing the horizontal vertica , 

P«.l clock divide (zoom, factors, or multiresolution p^irs 
of stereo images may be acquired for use with 
multiresolution hierarchical, stereo matching algorithms. 

initial ^ aCqUi ' ition - If s ° determined during 

initialization and setup, multispectral images are acquired 
simultaneously from image transducing devices, and 
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accompanying optical apparatus, with different spectral 
sensitivities, (eg. by using an RGB three chip camera) by 
using more than one analog-to-digital converter (in 136) 
simultaneously. Alternatively they may be acquired in rapid 
succession by using only one image transducer device 160 and 
either a set of different lighting conditions or a set of 
filters in the image transducing device 160. 

Multi-ri.x acqui.ition. Images obtained from a set of 
different viewing positions may be acquired simultaneously 
or in rapid sequence under software control of one or more 
parallel A/D converters and/or control of an electro-optical 
light guide switch or electronic switch that controls a 
choice of inputs to the A/D converters. Multiview 
acquisition is necessary for stereo imaging or for obtaining 
a time sequence of images from a set of viewing positions in 
order to use optical flow methods to study spatial geometry 
of an anatomic surface. 

Multi-iliuaiination. If so determined during 
initialization and setup, images of reflected or emitted 
light, are obtained under a set of different lighting 
conditions (e.g. structured light for 3-D surface 
reconstruction or ultraviolet stimulation to study tissue 
fluorescence) , in rapid sequence under software control of 
the lighting apparatus 154 and the image input module 136 
^ Calibration (314) : 

Once an image has been captured calibration 314 of 
the image is performed to calibrate for absolute distances 

du! t° rr"^ SPatia1 ' COl ° r ' ° r inte ^ty abortions 

due to the acquisition equipment and circumstances 

Spatial calibration. An image of an object of known 

Size may be us«d to calibrate for absolute, distance in an 

image obtained at a fixed or known working distance and 

magnification. For example, an image of a ruler or grid may 

be obtained during a calibration sessions or simultaneously 

wxth the image of the surface structure of interest 
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ruler or object of known structure and size may be 
automatically detected using this a priori knowledge with 
computer vision methods or manually indicated by the 
operator 330 using a pointing device 111. 0 nce detected 
5 distances between, or sizes of features of, this object 'or 
ruler may be computed by counting pixels. Then, given the 
known actual distances between, or sizes or intensities of 
these features, the absolute size of a pixel at this 
magnification may be computed and stored for subsequent use 
as a scale factor, which, when multiplied by the number of 
pixels of an image feature, results in an absolute distance 
or Si ze measure of the feature. A similar method to 
calibrate for image grey levels and nonlinearities thereof 
may be employed. 

15 colo* calibration. For color calibration, a color 

chart, or set of light emitting diodes, of standard known 
colors, may be imaged, either during a calibration session 
or during the acquisition of images of the surface feature 
under examination. if the color chart has a known structure 
such as vertical stripes, then its features may be detected 
automatically, using computer vision methods or 
semi-automatically with operator input (for example through 
a mouse or other pointing device ill) . Regions of the 
containing known colors may be used to identify the set of 
P^el values representing that color. This set of pixel 
values (e.g. reds green and blue pixel values) for- an 
individual known color may then be used to determine input " 
look-up table values, or pixel scaling factors to apply to 

:0 of Zr 1 : ° f " ^ - Wil1 " SUlt ±n ^ndardization 

of color between images obtained under similar lighting 

conditions. This procedure may also be used to calibrate 
color measurements to absolute wavelength if the wavelengths 
emitted or reflected by the calibration objects are known a 
priori. Numerous variations on this method of calibration 
5 may be employed by the invention. 
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Fixed background noise subtraction. Fixed background 
noise of an i m age sensor (for example due to inoperative 
Pixels in a charge coupled device array) m ay be obtained 
under appropriate stimulus conditions and digitally 
5 subtracted from each image. 

Distortion correction. Geometric distortions due to 
the image transducer devices including its associated 
optical apparatus, may be corrected using, for example 
spatial warping algorithms. 
10 Aspect ratio correction (computational) . Under 

circumstances where the aspect ratio has not already been 
adjusted to the ratio appropriate for a particular ■ 
^Plication (such as for an additional display device) it 

* i::o:r t hm d s justed using softwar * « — — 

Pr«*Proc«8iiag (316) : 

,« h «. Pre - pr ° CeSsin * 316 ot an image may be performed, if 
so determined by the initialization defaults and setup 
choxces, to transform the images to different color space 
0 representations (e.g. red-green-blue to 

hue-saturation-intensity or luminance-chrominances,. other 
types of image manipulation can be used to enhance or 
extract selected image attributes. [4,5] 

Col ~ -P— transformation.. Color video images are 
> -ually acquired as three separate red (R) , green (G * J 
blue (B, images. R— G— B images may be transformed to on f 
several alternative color space representations including 
for example, a Hue-Saturation-Intensity (HSI) perceptual 
color model or a luminance-chrominance model such as the 
W-Q model used for broadcast television. Hereinafter, the 
term vector image means a set of - images ,- for example, a 
color image represented in a 3-dimensional color space is 

T-irizi thre : " d ' 9reen and biue ±mages ° btai -* *«. - 

G B Camera - The terro image will refer to color 

components of an image, such as red, green, blue, hue 
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saturation, intensity, luminance or chrominance. m 
addition, the term scalar image may refer to black and 
white, gray scale, or binary images. 

Hua-satuorati.on-lnten.ity transf oraation . m one 
version of the HSI model the hue <fl) , saturation ( S ) and 
intensity (I> values for each pixel may ^ ^ 

red, green, and blue values as follows: 
(1) define R' - R - m ± n <r, G ,B) 

G' G - man (R, G,B) 
,,, ' B' - B - min (R,G,B) 

UJ if B' « 0, then H - (120*G') / (R'+g'+rm 

15 (4) I * (R+G+B) . " 

>t . 

Numerous variations on these models exist and any may be 
employed by the invention [6 ] . Thus a single R-G— B image, 
^th- a 3-component vector value for each pixel, may be 
transformed into a number of scalar images, each with a 
single scalar numerical value for each pixel— i e 
individual images displaying the hue, saturation, "or 
intensity of the i^age in addition to the obvious scalar 
images displaying the red, green, or blue components of the 
25 orxgxnal image. 

Muitiapactral g-nwralixation o* HSI t^an. formation 

The invention extends this model to higher dimensional 
• multxspectral images. For example, given a multispectral 
image with 4 components-R, G, B, and infrared (IR) ~ a 
"4-dimensional" generalized HSI model may be derived as 
follows: 

U) define R' R - min (R,G,B,IR) 

G' - G - min (R,G,B,IR) 

to/ " B ~ roin <R.G,B,IR) 

(2) if B ' I n ".-I* " u n,in ( ^' G ' B ' I R>^ then 
i t B , °' then H - A / (R'+G'+B'+IR' ) 
if IR - 0, then H - B / <R' +G ' + B' + IR' ' 

if " S' 5u en H " c 7 <r' + g' + b'+ir' ; 

40 Where A £ % 0 'g') ^ * " ° ' <*' +G ' +B ' +IR ' > ' 
B - f90* (B'+l) ] 
C - [90* (IR'+2) ] 
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D = [90' (R'+3) ] 

A, B C, and D may also take on other values in alternative 
embodiments of the invention. 

(3) S = (R'+G'+B'+IR') / (R+G+B+IR), 

(4) I = ( R+G +B+IR) _ 

. ^ h SP-Ctral ^e invention presents a novel 

method of obtaining a scalar image from an R-G-B (or other 
multispectral, image that estimates (subject to the spectral 
imitations of the apparatus that acquired the original 
-age) the scalar image that would have been obtained had 
the lma ge been acquired through a narrow bandpass filter 

21 \ PartiCUlar hUe - This is Performed by mathematically 
projecting the R-G-B color vector for each pixe l of an * 
ont o the R-G-B color vector of the given hue. since 
R-G-B color space may be modeled as a Euclidian space 
Projection of one vector onto another may be performed by 
obtaining the dot product of one vector on the other Thus 
given the R-G-B color vector of the filter with the des^d' 

ToTr 2 ^ (f ° reXamPl6yellOW <*=1, G =1,B=0>, cyan 

°' G " 1 ' B = 1 >' °r magenta (R=l, G =0, B=l) , the vector to 
scalar transformation may be performed as follows- 
F i,j - (R,G,B) i ^ j • (cl,c2,c3) 

= (R. .*cl) + (G. .*c2) + m *^-5\ * 

^' J . ±rl ' ( i,i c3) for ea ch pixel 

that * inn- original image. Note 

cnat * indicates multiplication. 

an \ r * T , hiS meth ° d PermitS simultan *°« acquisition of 

arbitrary number of multispectral images simply by 

IT W 3 S± T le R " G - B image " -Itispectral images 

the R Tl ' V " <<?r WaVel -^*> -ntained within 

tne R-G-B space . 

Surfac. and Sub«ur£,« Reconstruction <318) : 

35 316 mav^ ima9 *' ^ Callb " tion 314 *»* Pre-processing 
316 may be stored in the database 326 for later retrieval 
and/or may unaeroo immediate analysis 3 ^ 
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_ If an appropriate multiview or multiple illumination 
acquisition protocol was specified by the initialization and 
setup routines above, then these images may be used to 
reconstruct a 3-dimensional surface map of the anatomic 
surface in step 318 using one of several reconstruction 
methods. If, in addition, a temporal sequence of such 
^ages has been acquired, then a 4-dimensional (3^D space 
Plus time) reconstruction of the surface as it varies in 
tame may be obtained, by sequentially reconstructing a 
3-dimensional surface for each point in the temporal 
acquisition sequence. A 4-dimensional reconstruction of the 
surface may be used to estimate mechanical or motion-related 
properties of the surface. 

15 re CO n " Urf * Ca " C ° Mt ™ fci °»- * variety of 3-D surface 

15 reconstruction algorithms may be implemented by the 

invention if the appropriate images were acquired during the 
acquisition sequence [7] . For example, if stereo pairs of 
images were acquired, a variety of stereo matching and 
^angulation algorithms may be employed to obtain the 
elevation of the surface. Alternatively, other methods such 
as photometric stereo, structured light, shape-f rom-shading 
[8] moire, or optical flow may be employed if the 
appropriate lighting conditions and/or motion sequences were 
employed during image acquisition. 

One embodiment, of this system function 318 for a . pair 
of binocular stereo color images consists of the following 
steps: • . y 

(1) Obtain color space transformations (R-G-B to 
H-S-l an d v-!^, of tne original left and r . ght s ^ 

described in the pre-processing step 316. The following 
steps- may be applied to one or more of the nine resulting 
scalar images, i.e. R,G,B,H,S,I (intensity), y, i 
(chrominance), and Q. 

(2) Operate on left and right image pairs with one 
or more directional Sobel edge operators. 
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(3) For each of the resulting gradient images 
eliminate all pixels except those which are local maxima 
Then threshold the resulting images using a predetermined or 
computed pixel value, for example the midpoint of all 
possible pixel values - i.e. 128 for an 8-bit pixel. The 
resulting thresholded gradient images may be used as feature 
points for a stereo matching algorithm, such as the one 
described in the following steps: 

(4) . For each stereo pair of thresholded gradient 
images determine the location of corresponding epipolar 
lines in the left and right images. Place a window of 
predetermined size on the first feature point of the' first 
epipolar line in the left image of a stereo image pair. 

(5) Then search along the corresponding epipolar 
line in the right image using a sequential similarity 
different algorithm (SSDA) to minimize a distance measure, 
V between one or more stereo pairs of left and right 
thresholded gradient images, within a window, w, of size, n 
by m pixels, as follows: 



m n 



D " " . , . (f l (i-j) " F U, j)) 2 / V 

where F L is the left image, F R is the right 
image, V w i s the variance within the window w, and i and j 
are the horizontal and vertical pixel post on indices within 
the window. 

(6) Determine the window location ' along the right 
epipolar line where D w is minimum. 

(7) if the minimum D w is below a predetermined 
matching threshold, then consider the pixels in the center 
of the window in the left and right images as a pair of 
matched points. if the minimum D w is above the 
predetermined matching threshold but below a predetermined 
ambiguous match threshold, then increase the size of the 
window and repeat steps 5 to 7 until either a match occurs 
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or until the minimum D w is above the ambiguous match 
threshold. 

(8) Shift the left window to the next feature point 
along the left epipolar line and increment to the next 
5 epipolar line pair, if necessary, and repeat steps 5 to 7 
until the SSD algorithm has been applied to all feature 
points in the left and right images. 

<9) Triangulate to obtain the 3-D surface elevation 
for each pair of matched points. 

10 (10> Interpolate the resulting sparse 3-D dataset if 

necessary. 

Subsurface r«con*trucfcion . Since the depth of 
penetration by light into most biological surfaces is a 
function of the wavelength of the incident light, 
15 multispectral imaging may be used to estimate the depth 

and/or density of various subsurface structures or processes 
(e.g. erythema). First, an explicit or implicitly assumed 
model of the surface with its subsurface structures and/or 
processes is considered. Based on this model even if only 
implicitly assumed, and based on additional a priori 
information and/or assumptions, an algorithm which 
approximates the estimation of depth, i.e. subsurface 
extent, and/or density of a structure or process, may be 
derived .using a functional representation of this 
25 mathematical model, a linear or nonlinear combination of 
images, each representing light of a limited bandwidth of 
wavelengths, reflected from the surface and its subsurface ' 
layers and components, may be employed to approximate an 
image whose grey-scale values linearly represent a 
particular depth, density, or other characteristic of a 
subsurface structure or process. 

As an oversimplified, sample embodiment of such a 
method, consider, for example, a simplified model of the 
optical properties of skin wherein (1) light is attenuated 
exponentially as it penetrates through the skin and wherein 
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(2) melanin is of uniform density. . Then, subject to the 
assumptions of the model and other implicit assumptions, the 
depth of melanin for each pixel may be approximated by 
subtracting the log of an image, obtained (or computed) at a 
short wavelength, from the log of one, obtained (or 
computed) at a longer wavelength. This is equivalent to 
taking the log of one image divided by the other. This may 
be performed sequentially for a set of multispectral pairs 
of images.. For example, given images of skin obtained 
simultaneously through infrared, red, green, and blue 
bandpass filters, the log of the blue image may be 
subtracted from the log of the infrared image to obtain an 
image estimating the thickness of melanin in a subsurface 
region of skin that extends between the superficial depth of 
15 penetration of blue light and the deep penetration of 
infrared light. The thickness (or possibly density) of 
pigment contained in sequentially thinner subsurface regions 
may be estimated (subject to the implicit assumptions) by 
(i) subtracting the log of the green' image from the log of 
the infrared image to obtain a medium thickness subsurface 
estimate, and (ii) subtracting the log of the red image from 
the log of the infrared image. 

Using the method of spectral projection described 
above under pre-processing 316, a potentially infinite 
number of images, each computed for a different wavelength 
in the visible spectrum, may be calculated. Via sequential 
log subtraction of these images from the log of the infrared 
image, a series of images representing the depth (and/or 
density) of pigment in subsurface slices of progressively 
increasing depth with decreasing thickness may be obtained. 
The.se may then, be further manipulated to estimate pigment 
depth and/or density, or to estimate features of other 
subsurface structures or processes. 

Alternative embodiments of the above method may be 
35 obtained by employing a more accurate mathematical model of 
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the attenuation, scattering, and reflection of light of 

various wavelengths by a biological surface, such as the 

skin or retina. 

Image Segmentation (320) : 
S Segmentation is the process by which an image of a 

surface is segmented into two or more regions, for example 
regions representing artifact or interfering structures 
which must be detected and eliminated from subsequent 
analysis, regions of interest which will subsequently be 
analyzed, regions of . background surface, and subregions 
within the regions of interest which will be eliminated from 
subsequent analysis. Segmentation may be applied to 2-D, 

3-D, 4-D (space + time), R, G.BHSIyt / « ■ 

r rv, u,u,«,i,,x,Y, I, and/or Q images or 

any scalar or vector spectral derivative thereof. 
15 .When the invention is applied to the human cutaneous 

surface in vivo, the resulting images of skin may be 
segmented into regions of lesions and background skin. 
. However, hairs may- interfere with this part of the 
segmentation process, and may need to be detected and 
20 eliminated from this part of the segmentation process. m 
addition specular reflections within regions representing 
lesions of interest may also require detection prior to 
subsequent quantitative feature extraction. The borders 
outlining these various image segments may also be computed 
during the surface segmentation process 320. Thus the 
segmentation process may include one or more of several 
steps, which may be predetermined by the initialization 
routine or. chosen by the operator 330 during the setup 
routine or during the analysis if so required or desired. 
XiMxoxi d«t«ctioxx: 
. The segmentation of an image into regions, of interest 

(e.g. lesions) and regions of background surface may be 
performed in a number of ways. m the present invention, a 
Preferred method is a novel bimodal histogram segmentation 
technique, which employs the following steps whereby a vector 
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image is converted first to a scalar image (if the image is 
not already scalar), then to an intermediate threshold 
image, and finally to a binary image indicating the 
segmentation : 

5 1) Transform the vector R-G-B color image to one or 

more scalar images of a color space variable (e.g. 
intensity, hue, saturation, luminance, chrominance, red, 
green, or blue), as described in the preprocessing step'316 
above . 

10 2) Split the scalar images into 2-dimensional 

windows of specified dimension with a specified overlap. 

3) For each window, perform a bimodal threshold 
determination comprising a) computing a histogram at a 
specified bin size, b) smoothing the histogram, for example 
with a robust nonlinear smoother such as "53H twice" (which 
sequentially convolves the histogram with a 5-point median 
window, followed by a 3-point median window, followed by a 
3-point Hanning window, and then repeats this sequence a 
second time, c) computing derivatives of the histogram to 
identify any maxima and minima, d) checking if 2 maxima 
exist (i.e. the histogram is bimodal), separated by at least 
a specified minimum distance, and if so, setting the 
threshold for that window to the value of the lowest minimum 
between the maxima, and if not, leaving the threshold for 
that window undefined. 

4) If less than a specified mimimum number of 
thresholds (bimodal histograms) were found; change the 
window dimension and repeat step 3. 

5) For each window which was found to contain a 
bimodal histogram and for which a threshold value was set, 
assign the • threshold value- to the' center pixel of that 
window . 

6) For each window that was not found to contain, a 
bimodal histogram and for which a threshold value was not 
set, assign a value to the center pixel of that window by 
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using a two-dimensional interpolation between the windows 
with threshold values. 

7) Assign a threshold value to all pixels that w ere 
not wxndow centers by using bilinear interpolation. 
5 8) Compare each pixel in the resulting image of 

thresholds with its corresponding pixel in the original 
scalar image. If the threshold value of the pixel is 
greater than its original value, assign that pixel a value 
of 1 in a binary image. Otherwise, assign that pixel a 
10 value of 0. 

This segmentation method can be modified in several 
important ways to suit a given application. First, the 
larger the minimum distance allowed between, bimodal edges 
the stronger the edge must be in order to be detected. This 
allows operator control of the edges detected. Second the 
smaller the window dimension, the more detailed the 
structure emphasis. Finally, the higher the histogram bin 
value, the greater the number of bimodal histograms that 
«11 be found, accompanied by a decreased precision. The 
dimension of the windows can be adjusted depending on the 
size of the structure to be segmented. However, if the 
window dimension is lowered, the minimum distance allowed 
between peaks and/ot the histogram bin size must be 
increased. 



25 



Values computed during the segmentation routine are 
used to characterize the image. For example, the average 
distance between bimodal peaks is a measure of the edge 
strength. Another measure of edge strength is the 

30 TnTlir 0 *^* 11 ° f l0W ^ ° f two maxima 

and the value of the mimimum between the two maxima. m 

1 tlTT 3 ; ^ StrSngth ^ ^ ^ified by scoring 

the bimodal histograms during segmentation. 

! _ Mor P*°logic operations [5] are used in combination 

j with the above bimodal histogram segmentation method to 

, selectively segment desired features . For example, a 
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orphologic closing operation can be used to segment rete 
ridges, nodes, black and brown dots, globules, and other 
parameters described in [3] . A morphologic closing 
operation can be used to segment rete pegs, pigment pattern 
5 regressions, other parameters described in [3], and the 

interior of skin lines. To perform the closing operation, a 
scalar image is first eroded using a disk of a specified ' 
radius, and then dilated using the same disk. The original 
image is then subtracted from this modified image, and the 
10 bimodal histogram segmentation method is performed. To 
perform the opening operation, the scalar image is first 
dilated using a disk of specified radius, and then eroded 
using the same disk. Again, the original image is 
subtracted from the modified image, and the bimodal 
15 histogram segmentation is performed. By use of the opening 
or closing operations prior to segmentation, features of 
interest can be made prominent, whereby the bimodal 
histogram method is able to segment these features. In the 
present invention, this technique is employed in texture 
20 analysis methods described hereinafter. 
Blob d«t«ction: 

(9) Perform morphological "blob" detection to 
further segment the binary image into distinct regions of 
contiguous pixels ("blobs") which are either below or above 

25 the threshold. Blobs with pixel values below the threshold 
represent regions of interest (i.e. lesions or foreground) 
whereas blobs with pixel values above the threshold 
represent regions of background surface and/or specular 
reflections within lesions (i.e. nonlesions) . This 

30 definition of foreground and background regions may be 
reversed as determined by- the" initialization and setup 
routines or by image features extracted by the analysis 
routines . 
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(10) 
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Count the number of blobs representing lesions 
foreground) and the number of blobs representing nonlesions 
(background) . 

(ID Determine the size (i.e. area) of each lesion 
o and nonlesion blob. 

(12) Eliminate all except the n largest foreground 
blobs representing lesions (satellite elimination), where n 
" 3 P redete ^«ed number set during initialization, setup 
or during a previous analysis step; or alternatively use a' 
10 predetermined size, rather than number, n, as the 
elimination criterion. 

(13) Eliminate (and record the location and extent 
of) all except the n largest nonlesion blobs 

IS' blobs l 14 l R6COrd l0Cati ° n SXtent ° f a11 ^lesion 

blobs W hxch are contained within lesion blobs. These may be 
used as candidate regions for specular reflections 

represent"! ^ ^ °' ^ f °«^ound blobs which 

represent lesions, for example by searching for horizontal 
and vertical binary gradients (i.e., binary transitions) 

Thus the image of the surface is segmented into 
regions representing regions of interest (lesions), regions 
of specular reflections within these regions of interest 
and regions representing "normal" background. 

The results of this segmentation method, when 
performed on more than one color space variable, may be 
combined for certain applications . 

Alternatively, steps 1-8 of the above "lesion 
detection" segmentation may be performed by computing global 
histograms of the number of pixels of each intensity (L 
other color space variable) and then automatically or 

^es a hl y H <USing mQUSS ° r setting the 

threshed at the nadir between the two modes of the 

histogram distribution if it is bimodal . 

£?££*E!: ati0n With P rior •liaination of .trust™. 
with, known a priori £»»tur««: "ructuraa 
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Surface structures with known a priori features may 
be detected and eliminated prior to the "lesion detection" 
segmentation step described above. For example, for the 
human cutaneous surface , hairs may interfere with 
5 segmentation. Therefore, a prior segmentation step designed 
to detect and eliminate hair, may be performed before lesion 
detection. As a sample embodiment of such a method, 
consider the following steps: 

Segmentation with prior hair elimination: 
10 (1) Given one or more color space transformations of 

the original image (Transformed Image), as described in the 
pre-processing step 316, 

(2) perform an appropriate grey-scale morphological 
closing operation (a set of ordered statistic operations) on 
the Transformed Image to obtain an estimate of an image 
without hair denoted the Less Hair Image. This closing may 
be implemented as the concatenation of a set cf grey scale 
closings, for example, four concatenated grey scale 
closings, each with a different structuring element. The 
structuring elements used in the closing operations should 
incorporate some a priori information about known features 
of human hair in images obtained at a given surface 
magnification. For example four linear structuring 
elements, each at a different orientation angle may be 
sequentially applied. -The maximum diameter of the resulting 
equivalent concatenated structuring element should, for 
example, be greater than or equal, to the width of the 
majority of hairs in the image. Note that under most 
circumstances this method results in better hair elimination 
for image regions where hair is superimposed over background 
normal skins- than- for image regions where hair overlies a 
lesion. 

The Less Hair Image may alternatively be obtained 
using a threshold-type of segmentation (e.g. "lesion 
detection" segmentation steps 1-8) applied to a hue 
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transformation of the original R—G-B image or to an 
-tensity transforation of the original KGB image followed 
by a color space projection onto the mean hue color ve 
for a known image region containing only hair (see 
Pre-processing 316) . 

(3) subtract the Less Hair Image from the original 



(4) Perform a "lesion and blob detection" 

oXT at : on (steps 1-14 ^ ° n the Hair - 

obtain a Binary Hair Only Image. 

(5) Perform a "lesion and blob detection" 
segmentation (steps i- 14 above, on the Less Hair Image to 
obtain a Binary Less Hair Image. 

" not in 111 J 1 " 13 WhiC * a " iD B±nary Ha±r — 

not -m the Binary Less Hair Image are considered to be hair 

external to the border of the lesion. 

(7) Pixels which are both in the Binary Hair T fflaop 

20 tit" Bi0ary LM " ^ ^ --i^re'd to be h L r 

^0 that is within the borders of the i^o- ^ 

, . eirs or the lesion and which ar-o -t-v,=,~ 

ICrir -ture II™ 

<S) Construct a Hybrid Image consisting of two 

23 IZZ'^T^ ^ 3 * mU * * -^3 Of the 

,2i < l **"' B * lr These edge, are obtained by 

applying blob detection step 1S above to the Binary Less 
B »"Ima,e. The region outside this edge border is replaced 

edge bor": 9 1 ^ ">* 

This resnltxng Hybrid Image thus contains an image without 

(9) The border will under most circumstances be a 
c osed contour which contains the true lesion border Within 
«. *ue to properties of the -lesion detection" segmentation 
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method above when applied to an image containing hair. Thus 
the final step consists of applying the "lesion and blob 
detection" segmentation method (steps i- 15 above) to the 
Hybrid image to obtain a more accurate estimate of the true 
5 border of the lesion. 

(10) The pixel locations determined in step 7 
immediately above denote the location of hair covering the 
lesion, and are used to eliminate these pixels f^om 
subsequent, feature extraction 322. 
10 Quantitative feature extraction (322) : 

Once the borders of lesions are obtained by 
segmentation 224, quantitative features of the images may be 
computed 22 6. y e 

r«atur«s o£ tiae Global Surface: 
^constructed „ap, of , surface may be counted using 

rnr t * in the " Mob 

3-14) described above 320. 

Location of le.ion.. The location of lesions on the 
surface may be graphically represented on a set of low 
magnification (global) views of the surface. These 
locations are determined by the 3-D location parameters 
nput by the position sensor 152 when a higher magnification 
(local, view of a lesion on the surface was acquired A 
straightforward geometric transformation may be applied to 
correct for the distance between the mounting pos^ion of 
the position sensing element 324 and the center of the local 
region of anatomic surface which was imaged i These 
coordinates are then mapped onto the global or reference 
xmage of the surface using data obtained during calibration 

the position sensor coordinates to the pixel location 
coordinates on the global or reference image. 

Alternatively, rather than use a global or 
reference digital photographic image, a standardized 
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synthetic body surface map (with graphical dis 
employed. Moreover, if this surface m ap can be 
Parameterized, then the locations of individual lesions, may 
be represented as quantitative values representing the 
location on the surface map. 

3-D and 4-D lurfac. morphology «d topology 

(or « d time varying 3-D surface map,, obtained during the 

318 above, may be quantified 
For example S urface elevation statistics (e.g. mean, mode 
variance, skewness, may be obtained via straightforward ' 
computation. Topological features, such as the number of 
peaks m surface elevation that are above a certain 

TrTaTt: COUnted; " g±Ven ^ b ^ ««« *>« of 

surface elevat.cn, the Euler number of the resulting binary 

surface, defined by this threshold, may be computed using " 

b"ary 'image processing methods (similar to the "blob 

detection- step described above), and may be used as a 

parameter describing surface texture. 

20 r**tux«« o* Local Surraca l^.ioa.: 

- the J- 01106 leSi ° nS ° n ^ anat ° mic surf ^e •« detected bv 
the segmentation algorithms 320, their morphologic and * 
spectral features are determined and represented as a set of 
quantitative parameters. Por surfaces such as the s^iL 
25- ^antitative. descriptors of lesion asymmetry, border 

^regularity color, diameter, elevation, and texture may be 
of clinical importance. y 
3fcsynm«txy : 

describ^' 1 " 3 ^" featUreS ° f 3 2 - 1 -»^«»l ^sion 
describe, its- asymmetry are roundedness (defined as the ratio 

of the minimum to maximum second moment of the lesion about 
; ; Centr0id > *«* -centricity (defined as tne a f 

the nuoor to minor axis of the lesion, . Roundedness is 

multiplied by 100 to present it as a percent varvina h\ 
100% fnrs „ * . petcenc varying between 

100% for . a perfect circle and 0% for a straight line. 
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Border irregularity. Border irregularity may be 
determined by the following parameters: 

(1) R adial or polar «^n.f^ ^~ rri a polar 

representation of the border (i.e. one that parameterizes 
5 the distance between the centroid of the lesion to the 
border as a function of the angle above the horizontal, 
statistics may be defined, for the distribution of this' 
radial distance, over all angles from 0 to 360 degrees For 
example, the standard deviation of this radius may be used 
10 to quantify radial border irregularity. 

2) C ° nve " deficiency »nH .- eeaa nf ^ K _„_ r ^ 
convex deficiency of the border quantifies the area 'of any 
concave indentations (scallops) into the lesion as an 

15 ^ S ° 1Ute meaSUrS ° r relative to lesion's total area. 

The convex excess similarly quantifies any conve- 
protrusions (pseudopodia, out of the lesion as a percentage 
of the lesion's area. 

One way to determine the convex deficiency of the 
border is a novel line fit method in which the convex hull 
of the lesion is first computed as follows: 

(i) Determine the point where the longest ray from 
the lesion centroid intersects the lesion border. 

(ii) Beginning with this point, compute straight 
l^nes which extend at an angle that is progressively 
incremented until, a line is obtained which intersects 
another pixel on the identified lesion border. 

(iii) Connect the points which intersect the border 
Uv) Repeat steps (ii) and (iii) until a closed 

contour is obtained or until the point identified in step 
30 <*> is reached. p 

• • " <the ^gle -increment in step,(ii), is small enough> 
n excellent approximation of the convex hull of the 

tL S1 borI b ° rder W±11 ^ ° btainSd - The —x deficiency of 
the border » then obtained by subtracting the area of the 
lesion (i.e. the number of pixels or square millimeters 
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Tull I"' 11111 fr ° m arSa Within *h- convex 

hull. The result may then be normalized by dividing it by " 

the area of the lesion and then multiply . ng it * y 

obtain a percent. 

5 comh - ^ ^ PrS3eilt inVenti ° n ' ™°-Phological methods are 

Wlth linS meth ° d - * —1 way to compote 

the conve, deficiency and excess in a way which allows f 
robu St comparison between the two. To compute the convex 
deficiency, the following steps are followed: 

1) The convex deficiency of the binary image of the 
lesion 1S determined using the line fit method. 

2) if the convex deficiency is it . • 
-age is dilated using a dis* of a specified radius, anT 

15 bl 2 ^ rSPeated - ^ ^ ~ deficiency 

becomes zero within a specified tolerance, the procedure 
moves on to step 3. 

,„ ^ • b±nary ±S er ° ded us "5 the same disk as 

20 the area'o/l 6 ±S ^ subtracting ' 

area of th T & " ^ ° f ^ *"» the 

area of the final binary image of the lesion. The result 

«y then be normalized by dividing the result by the area of 

25 T Y l ToT\" Ln&rY ^ th6n """" ^ Multiplied 

by 100 to obtain a percent. 

with tlTllT™ " ■ d *" t - i »«' .i-ilar steps. 

d "ff eren ° e « tot f "" «» i» successively • 

30 eLal „ul J""' ^ " " ^ <W.t«, .„ 

=*t«c V „ g the are, of the final binary im a g . of tne iLion 
^ tne area of tne initial binary i„ g e c f tne lesion. 
The result „ay tben be nOEmali2 . d fay divid ^ 

3= ^00 in±tial " d the ° -"iplied by 

100 to obtain a percent. 
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Inverse compart-^ A parameter related to the 

compactness of a lesion's border is defined using the s^e 
of the perimeter of the border divided by the area within 
the border tio,. This dimensionless number is approximately 

12 for a perfect cxrcle and rapidly increases for borders 
with finger-like extensions. 

[ A) /"ctal di wm Hon -n. fractal dimension, fd, of 

grid Z .I* eStlmated ^ firSt ™**>«*P°^* - rectangulL 
over the image ,nj. The size of boxes defined by the 

Lt N m ; y Sma11 " 3 Singl€ PiXfil " ~* be 

£t N be defined as the number of grid boxes which contain 

wilt? 7 leSi ° n ' S b ° rder - ^ bS - 

; a . Slngle 9r±d bOX divide « the maximum diameter 
of the le Saon . Then the fractal dimension of the border is 
estimated by dividing the log of N by the log of l/ t 
Estimates of the fractal dimension may be obtained for more 
than one grid size and then averaged. 

<5> P i3 tinctne.s of the horror -Parameters, which 
are related to how distinct the border is relative to the 

estimated using a variety of statistics, including 
statistics derived from gradients of i mage intensity, hue, 
enelgy ratl ° n —/or 

A preferred method of characterizing the border 

TocTlTrT 3 ^ ^ aPPly 3 gradient °**^™> Preferrably 
local variance, within a border area. Statistical 

Properties of the border gradient can be used to describe 
the edge strength of the border. scribe 

the k ' ^ StrSngth Can als ° quantified by scoring 

the blmodal histograms during segmeritatio ^ For . y 

the . distance between peaks P e 

the edge strength. Another measure of edge strength is iL 

^Z^:^— 1 - ^ lower of the Two mlx^a 
e value of the u,u.„ between the two maxima. 
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The Fourier domain energy of the border can be used to 

characterize border convolutedness . Arithmetic combinations 
f the Fourier coefficients computed from the binary image 
5 can be used for this quantization. Alternatively, the 

Fourier domain power of the border can be computed by ta*in« 
the 2-dimensional Fourier transform of fche im l g t ak a Z 9 

then integrating the sqU are of the result over the Folrie" 
domain space. ourier 

of a les- COl ° r ° f ••<* within the border 

of a lesion may be quantified by its hue f«, * n v, ^ 
to -?fin (as a nun »t , er from 0 

to 360 degrees, or, if properly calibratort 

in „,„ . 4 * F y calibrated, as a wavelength 

" nanometers) and by its saturation (as a percent „ 
and 100*1 r«i . 3 percent between 0 

na 100%) [61. The distribution of colors within the 
15 identified boundaries of a lesion m»„ k 

novel w,v « ' s or a lesion may be represented in a 

Tl L7 J 3 3 ° f P ° intS ° n 3 COl ° r Wheel ^-Plays 
all possible colors (of the given color space model) at a 

pL V ;rs V L 1 .° f ±ntenSity; SUCh " — or the 

20 pLel ell ^ l6Si0n b ° Undary - Th6di ~c f 

P«.l colors within the lesion border may also be displayed 
as^ndividual histograms of hue and saturation, o^fa ' 
2-dimensional joint histogram of hue and saturation 
Quantitative parameters may be extracted from these' 
graphical presentations of color distrifani-<«« • , 

TZtt SPeC ° lar " fle °«°" or lnt.rx.ri*, 

30 ool • *"* """^ ° f 1 "* 9e a =*» isi "°° w±»J oil a„d 
T t0 redU ° S ape = ul " ""ecticn described 

anllvLT " ^ Partic » 1 "^ appropriate „„e„ co!or 
analysis is to be performed. 

Diameter and Ar Sa — Tne diameter of = t 
35 be computed aiameter of a lesion may 

computed as the maximum and minimum diameter through the 
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=»-tro±d of the lesion. The ar6a is 

by the V FiXelS " ithi " b0rd " "* -Itiplyin, 

by the scale factor determined in the spatial calibration 
sequence above. on 

5 Elevation — if 3-dimensional surface 

recon S truction was performed above 318, then quantitative 
Palters that describe features of the 

surface may be computed such as statistical features of 
r face el , vation (i . e . mean elevat±on/ stan - o* 

weTr:r;is k : rtosis ' and other ™ s ° f — ??• 

-11 as histogram and spectral density of the elevation) 
Other morphologi c feai-nroo 

rr«sr- —a - - 

TSXtUre " A vari *ty standard and novel 

region Ihls te *ture an.yi s i s „ ay be fl J 

""f' S ' "" i0h inCl ° d " <*«acteri 2 atio„ of pi^nt 
Pattern, distribution, homogeneity, regression. *L other 

»). other , !ss may include ln skin line 

analyse or temporal or apati.l. characterization of ' 
capillary patterns. In ,ag.s obtained using oil and/or 
perpendicular polarizers are used in u. 

Hillary pattern, analysis ^ t ££T ™. . 

re, ti .e inde, „ ianatch ^ - 

the sfcl „ llne and s k in scaling analysis, images oht, ned 
oil and/or of perpendicular polarizer i maf ro« J 

— de o t enBa „ ce the imag / of th .-™:^ M r " 

One method used to characterize the teature of an 
3= -age is computation of the spatial gray le.el dependence 
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° r SeC ° nd ° rdSr hist °9- (probability density 

such as autocorrelation, covariance, inertia, energy 
entropy, local homogeneity , contrast , and ^ ^ 

iTTiiii z : a novei variation ° f this ™' ^ 

«* be split lnto Windows of a spec . fied 

loc T can be coraputed for each window - «*• tL 

local variance of these parameters can be computed to 
provide further texture characterization. 
10 bother method used to characterize the texture is 

the computation of the run length gray level matrix U2J 
From the matrix can be computed parameters such as short 



split up into windows to obtain the local variance of these 



15 

parameters. UI In «e 

Another method used to characterize the texture is 
the examination of the extrema density. m this method the 

nela^b ^ " C ° mpared to a ^ified set of its 

neighbors along that line to determine if that pixel is a 
maximum or minimum in that set. if it ±. it ± ! m \ ! 
such fpy. j _ . 13 ' xt is marked as 

«ch. Thrs can be repeated for a specified set of angles 
^e., 1( o, 45 , 90/ mJ 135 degrees ^ resultin f"' 

a mnlmra ln all ±mages , s ^ resulting 

ZTs m :r - ^ d istributi :: : ?:tr g 

^ "° »* <*««=terired o y calculating 

* : ™ ri "~- «<» «"« — nts of the „*. r of 
extrema m a given region of the image. 

is the alt "°" er Keth ° d " the texture 

^.ge of a \ " ^ fraCtil diB,nS " n ° f « ' 

^sfra ot I H S1 ° n - * Pref "" d ~ttad for approximating 
this fractal dimension is to first superimpose a 3- 
di-ensional grid of ooxes over a 3-dimensicnal mathematical 
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representation (z = t(x , y)) where 2= the pixel valug ^ ^ 
location <x,y) of the scalar image, the length of the sides 
of. the grid boxes being greater than or equal to the pixel 
size of the image [11]. Then, the number of grid boxes 
5 which intersect the surface of the 3-dimensional 

representation of the scalar image is computed. The fractal 
dimension is approximately the ratio of the logarithm of 
this value to the inverse of the length of the side of the 
grid boxes normalized to the maximum diameter of the lesion 
The computation can optionally be performed for several grid 
box sizes, and the results averaged. 

Yet another method used to characterize the texture 
is a Fourier domain fractal dimensional analysis The 
present invention uses a 2-dimensional extension of a method 
15 described in [13 ] . in the extended method, a scalar image 
" first split into a set of windows of a specified overlap 
Each window is convolved with a 2-dimensional data tapering 
window, for example, the following novel 2-dimensional 
extension of a 1-dimensional Hanning window: 



20 



25 



The windows may then be zero-meaned and unit- 
normalized. A Fast Fourier Transform of each window is 
taken, from which a power spectral density P(u,v) is 
determined. The fractal dimension D may be determined using 
a least squares method on [11]: 



30 



35 



The texture can be further characterized by examining 
parameters such as the variance of D. 

Yet. another method used to . characterize the texture 
" M anal y si * of the power spectrum of an image or image 
region, computed, for example, from a Fourier domain 
representation of a scalar image, or from a variety of other 
parametric and non-parametric methods [14]. Many features 
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can be extracted from the 2-dimensional power spectrum of an 
image.- m a preferred method [15], the. amplitude spectrum 
is integrated over annuli or radii sectors and parameters 
such as the variance or other measures of the power in the 
sectors is used to characterize the texture. Features can 
also be extracted from the phase spectrum, treated in a 
s-imilar manner. 

Yet another method used to characterize the texture 
is an analysis of Nth order probability density functions 
In this method, a scalar image is first split into a set of 
windows of a specified overiap. For each window, the 1st 
order probability density function is computed. From this 
function, such parameters as the mean, variance, skew 
kurtosis, energy, and entropy of the function can be 
determined. These parameters can be used to create an image 
transform applicable to Nth order probability density 
function signatures. The texture can be characterized by 
such parameters as the mean, variance, and higher order 
moments of the probability density function signatures. 

The texture in the area of any detected edge, for 
example the border of a lesion, is characterized by first 

"xtuT^ 010 " 1 " 1 ° P " atiOD3 and computing any of the 

tettur. parameters described above. In a preferred method, 

5 ^2 96 ±S er ° ded and/ ° r d±lated USin * * of a 

* reified radius. These images are subtracted frbm the 

IZllTnaT^^- re3UltS " e ****** ° f th. 

inner and/or outer edge regions. One or both of these . 

regions can be used as the region of interest for texture 
characterisation, using any of the methods described 
hereinabove . 

The- present invention employs a method for 

^rt^r 1 " ^ ^ teXtUre ' FirSt ' « edge transform is 
performed on a scalar i mage to detect an edge. The edge 
transform can be the standard Sobel, Roberts, or variance 
transforms/ or it may be a morphological edge detector ■ 
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Next, the image is split into windows of a specified 
overlap. The number of edge pixels in each window is 
computed. Parameters such as the mean and variance of the 
number of pixels in each window are used to quantify the 
edge texture. This method can be generalized to any of the 
feature extraction methods of the present invention for 
computing the local spatial variation in the occurrence of 
those features. 

Nth Order Moment Distances Between Images. The Nth 
order moment distances between any scalar images are 
calculated. In a preferred method, the moment analysis 
described in [16] is used to compute rotationally and 
translationally invariant Nth order moments for binary and 
corresponding scalar images. The difference between the 
moments can be used to characterize many features such as 
color asymmetry. Multispectral features. A 

multispectral color wheel using the multispectral 
generalization of the HSI color space transformation, 
described in pre-processing 316, may* be used to plot 
generalized multispectral distributions of hue and 
saturation and to calculate statistics thereof. 
Multispectrally derived features, such as relative lesion 
depth and volume, may be computed using information derived 
from the subsurface reconstruction method described above in 
25 system function 318. Estimation of such features may be 

improved further by incorporating information about surface 
elevation derived from the surface reconstruction methods 
also described above in system function 318. 

A variety of other spectral algorithms, related to 
30 algorithms used for example to detect relative amounts of 
various substances with known spectral properties (e.g. 
hemoglobin, melanin, bilirubin in skin) may be applied to 
multispectral images of visually accessible anatomic 
surfaces . 
35 Classification (324) : 
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Given, a set of quantitative features of lesions or 
other structures or processes on an anatomic surface, these 
features may be used to classify the lesions, structures or 
processes into one or more classes. standard methods of' 
classification, decision making, or clustering may be used 
depending on the type and amount of a priori information ' 
available about each of the potential classes and about each 
of the lesions, structures, or processes on the anatomic 
surface [9J . 

10 w F ° r e2sam Pl^ given some a priori information 

about how morphologic and spectral features of pigmented 
cutaneous lesions correlate with microscopic pathologic 
features thereof, a classification method may be employed by 
the invention that incorporates this a priori information 
into a classification scheme that would estimate the 
probability that a given cutaneous lesion belonged to a 
particular pathologic class or diagnosis. 

Or, for example, consider multispectral data, and a 
prion information about the known spectral properties of 
certain pigments and other structures in the skin, then, a 
classification scneme could be employed by the invention 
that would classify each pixel as belonging to one, or 
possibly more than one, class representing, for example, 
melanin, oxyhemoglobin, deoxyhemoglobin, bilirubin, or 
*o nonpigmented, skin. 

Numerous variations on these methods and 
classification schemes may -be implemented by the invention 
Databaaa (326) : 

0 are h .° DCe leSi ° nS ° n ' " re9i ° nS ° f ' the atomic surface 
are detected by the segmentation algorithm 320, their 

™T°IT C ; SPeCtra1 ' ^ ° ther featUreS " e determined — 
represented as a set of quantitative parameters 322. These 

quantitative parameters are stored in a database 326 along 

with the original and processed images, all acquisition and 

Processing parameters, and clinical data provided by the 



20 



:; * Substitute sheet 



WO 90/13091 



PCT/US90/0201S 



10 



15 



- 43 - 

patient and physician. Information in the database 326 may 
be used for subsequent temporal comparison, tissue 
classification, or clinical decision making. 

The database 32 6 automatically records all protocols 
and parameters used during image acquisition 312 r processing 
314,316,318, 320,322, 324, storage 326, display 328, and if 
used, control 332. It thus stores all internal parameters 
describing where each image is stored. It also stores a 
miniature version of each image, called an image icon. Tne 
database 326 does all of the internal record keeping 
necessary for efficient storage, analysis, and display. 
Data, including all images, may be recovered from the 
database by the operator 330 using a graphical screen 
interface with a mouse 111 or other input device, e.g. 131, 
with a structured query language, or via other software 
means . 

A sample embodiment of a database structure 326 
includes three main types of data records — patient 
records, acquisition records, and image records, A patient 
record contains all clinical information about the patient 
and points to all acquisition recoxds associated with the 
patient. An acquisition record contains all acquisition 
information and points to all images taken during a 
particular acquisition session. An image jrecord contains 
25 (l) filenames and locations of all images associated with 

the record, (2) all extracted features of tne. images in the 
record along witrh the particulax choice of methods and 
parameters used to obtain them, and (3) location with 
respect to global or reference image of each image in line 
30 record. 

Diaplayr mnd. liaa*dcopy (328) : ^ 

Various types of 2- or 3-dimensional video or grapbic 
displays may be employed to display nigh resolution, true 
colors and/or multispectral images and extracted features 
thereof. A liquid crystal stereoscopic display may be used 
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*o display 3-D data or stereo images, or, £or ej:ample a 
reconstructed 3-D surface may be rendered and displayed as a 
shaded object viewed from an arbitrary perspective or 
displayed as a profile or pseudo three-dimensional diagram 
or representation or attribute. Holographic or 

w \T~iT ted holographic displays may — - -™ 

Simultaneous visual presentation of a set of global 
reference images may be displayed with a superimposed 
graphical map of the location on the surface of local 
images This map may be color, graphically or numerically 

s Tt 3 ° f ima9e iC ° nS ° f l0Cal im ^ es ^ '-re 

33 Tm dl8plWd ° n thS Same — n. The operator 

330 may select an image icon, using a pointing device in 

to ^f^^ **1 version of this image,' 

to control processing and analysis, and to display any or 

Includ 1 ^ featUr6S tber60f ' " additi ° n ' if the 

local ° f eithSr l0Cal " gl ° bal imageS ' eith - 

local or global image icons can be used to retrieve 

ZZZTT'l 10031 ° r 9l ° bal imageS ' ^ ±COn is 
related to its corresponding image by means of a visual 

method of coding (e.g., color-coded box, color-coded border 
or numerical coding) . oorder, 

A hardcopy unit 146, such as a continuous tone, hiah 
resolution, digital color printer, may be used to obtain a 
hardcopy of any screen displayed on the monitor 144 
including images and data. 
D*ei«ioa; £ control (332) : 

Having thus obtained images of anatomic surfaces and 
quantitative features and classifications thereof, this 
-formation may be used either directly by the invention, 

3SSiSt Clin±Cal deCiS±0n •»« patient 

management, < 2) assist in other decision mafcing, sLh " the 
choice of a cosmetic of the appropriate color, or (3 ' " 
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provide a control signal input to ah instrument or apparatus 
such as a computer-controlled laser angioplasty device. 
Assistance to a physician may be rendered not only by 
supplying quantitative information about lesions on the 
surface but also by providing information in the form of 
stored images and clinical data of the appearance of the 
lesion in the past. 

Having shown the preferred embodiment, those 
skilled in. the art will realize many variations are possible 
which will still be within the scope and spirit of the 
claimed invention. Therefore, it is the intention to limit 
the invention only as indicated by the scope of the claims. 
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Claims 

1. A method for characterizing a region of a surface 
of a person's body, comprising ° e 

of th r6C ! rding 3 ,roltl -P« ct »i digital image of a portion 
of the surface that includes the region, a*d 

quantitatively extracting a selected feature from 
said multispectral digital image. 

2. A method for characterizing a region of a 
person's skin, comprising 

applying a moistener onto the skin region, and 

that i " C ° rdiDg 3 dig±tal ima * e *«» * Portion of the skin 
that includes the region. 

3. A method for characterizing . region of a 
person's skin, comprising 

directing a source of illumination through a first 
polarizer onto the skin region, 

from th r T BJ 3 di9ltal ±mage thr ° Ugh a Second Polarizer 
from the illuminated skin region, and 

quantitatively extracting a selected attribute or 
-presentation or a selected feature from said digital 

4. A method for monitoring a region of a surface of 
a person' s body, comprising surface of 

\ eCOrd±ng * time a first multispectral 

tixgxtal-xmage of a portion of the surface that 
includes the region, 

• recording at a subsequent time a subsequent 
multispectral digital i^age of a portion of the 
surface that includes the region, and 

• 5 A C °T aring Said f±rSt and subsequent images. 

rule for'cia T * ««•*»! decision 

rule for classification of pigmented skin lesions 
comprising ' 
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constructing a set of data comprising selected 
features quantitatively extracted from multispectral digital 
images recorded from a plurality of skin lesions, by 

recording a multispectral digital image of a 

portion of the skin surface that includes each such 

lesion, 

quantitatively extracting a selected feature 
from each said multispectral digital image, and 
establishing a diagnostic decision rule based on a 
comparison of said quantitatively extracted selected 
features from each said multispectral digital image with an 
accepted clinical diagnostic tool for said skin lesion from 
which said image was recorded. 

6. A method for classifying a region on a body 
surface, comprising 

recording a multispectral digital image of a portion 
of the region, 

quantitatively extracting a selected feature from the 
multispectral digital image, and 

applying a decision rule to said selected feature. 

7. A method for estimating the depth of a skin 
lesion having a structure in three dimensions, comprising 
illuminating the skin lesion and measuring the light 
reflected from the lesion using multispectral imaging means 
which concurrently acquires a multispectral image. 

8. Apparatus for monitoring a region of a surface of 
a person's body, comprising 

global digital image acquiring apparatus, 
local digital image acquiring apparatus, and 
means -for establishing the position of a local 
- image acquired by said local .Imaging apparatus on a 
global image acquired by said global imaging 
apparatus . 

9. Apparatus for characterizing a high resolution 
image of the surface of a person's body, said apparatus 
comprising image recording means- having a- sufficiently great 
working distance from said surface that said means is 
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capable of recording in a single image at least a 
substantial portion of the body surface. 

10. Apparatus for characterizing a region of a 
person's skin, comprising 

means for directing illumination from a source 
through a first polarizer onto the skin region, 

means for recording a digital image through a second 
polarizer from the illuminated skin region, and 

means for quantitatively extracting a selected 
attribute or representation or a selected feature from said 
digital image . 

11. Apparatus for monitoring a region of a surface 
of a person's body, comprising 

means for recording at a plurality of times a 
plurality of multispectral digital images of a 
portion of the surface that includes the region, and 

means for comparing said images. 

12. Apparatus for forming a diagnostically useful 
decision rule for classification of pigmented skin lesions, 
comprising 

means for constructing a set of data comprising 
selected features quantitatively extracted from 
multispectral digital images recorded from a plurality of 
skin lesions, by 

recording a multispectral digital image of a 
portion of the skin surface that includes each such 
lesion, 

quantitatively extracting a selected feature 
from each said multispectral digital image, and 
means for establishing a diagnostic decision rule 
based on a comparison of said quantitatively extracted 
selected features from each said multispectral digital image 
with an accepted clinical diagnostic tool for said skin 
lesion from which said image was recorded. 

13. Apparatus for classifying a region on a body 
surface, comprising 
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means for recording a multispectral digital image of 
a portion of the region, 

means for quantitatively extracting a selected 
feature from the multispectral digital image, and 

means for applying a decision rule to said selected 
feature. 

14. Apparatus for estimating the depth of a skin 
lesion having a structure in three dimensions, comprising 

means for illuminating the skin lesion, and 
multispectral imaging means whiich concurrently 

acquires multispectral images for measuring the light 

reflected from said lesion. 

15. Apparatus for monitoring a region of a surface 
of a person's body, comprising global digital image 
acquiring apparatus, said global image acquiring apparatus 
being capable of recording a high resolution image. 

16. Apparatus for performing quantitative feature 
extraction from an image that includes an image of a lesion, 
comprising means for employing statistical properties of the 
distribution as a function of angle of the distance between 
the centroid of said lesion to said border as a 
quantification of the border irregularity. 

17. Apparatus for performing quantitative feature 
extraction from an image that includes an image of a lesion, 
comprising means for employing the convex deficiency or the' 
convex excess of the border as a quantification of the 
border irregularity. 

18. Apparatus for performing quantitative feature 
extraction from an image that includes an image of a lesion, 
comprising means for determining or estimating the fractal 
dimension of the border as a quantification of the border 
irregularity. 

19. Apparatus for performing quantitative feature 
extraction from an image that includes an image of a lesion, 
comprising means for determining the distinctness, of the 
border as a quantification of the border irregularity.- 
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I 20. Apparatus for performing quantitative feature 
extraction from an image from the skin that includes an 
image of a lesion, comprising means for quantifying the 
color distribution in an image or region of an image. 

21. Apparatus for performing quantitative feature 
extraction from an image from the skin that includes an 
image of a lesion, comprising means for quantifying features 
of the surface elevation of a body surface. 

22.. Apparatus for performing quantitative feature 
extraction from an image from the skin that includes an 
image of a lesion, comprising means for reconstructing from 
saxd image the surface elevation of a body surface. 

23. Apparatus for performing quantitative feature 
extraction from an image from the skin that includes an 
image of a lesion, comprising means for characterizing the 
texture cf any scalar or vector image or image region. 

>24. A method for estimating the depth of a skin 
lesion having a structure in three dimensions, comprising 
illuminating the skin lesion, and 

measuring the light reflected from said lesion using 
measuring means which concurrently acquires multispectral 
images . 

25. - The apparatus of any of claims 9 through 14, 
further comprising image acquisition apparatus. 

26. The apparatus :of any of claims 8 through 15, 
further, comprising a visual display. 

27. The apparatus of any of claims 8 through 15, 
further comprising means for quantitative feature 

. extraction. 

28. The apparatus of claim 8 or 25 wherein said 
image acquisition apparatus comprises means for acquiring 
images of the body surface at a plurality of magnifications. 

29. The apparatus of claim 8 or 25 wherein said 
image acquisition apparatus comprises means for acquiring 
images of the- body surface under a plurality of spectral 
conditiJgns . , : 
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30. The apparatus of claim '8 or 25 wherein said 
image acquisition apparatus comprises means for acquiring 
images of the body surface from a plurality of views. 

31. The apparatus of claim 8 or 25 wherein said 
image acquisition apparatus comprises means for acquiring 
images of the body surface under a plurality of lighting 
conditions . 

32. The apparatus of claim 8 or 25 wherein said 
image acquisition apparatus comprises a light source and an 
image transducer and means for positioning a first 
polarizing filter between said light source and the body 
surface and means for positioning a second polarizing filter 
between the body surface and said image transducer. 

33. The apparatus of claim 8 or 25 wherein said 
image acquisition apparatus further comprises means for 
positioning a transparent member over an oil -covered body 
surface . 

34. The apparatus of claim 33 wherein said 
transparent member comprises a flexible membrane. 

35. The apparatus of any of claims 8 through 15, 
further comprising means for preprocessing the image or 
image regions, wherein said preprocessing means comprises 
means for performing a color transformation of each pixel of 
an image according to a three-dimensional color model. 

36. The apparatus of claim 35 wherein said color 
transformation is generalized for multi spectral images of 
greater than three dimensions . 

37. The apparatus of any of claims 8 through 15, 
further comprising means for preprocessing the image or' 
image regions, wherein said preprocessing means comDrises 
means for spectral projection, whereby a scalar- image for a 
given hue is obtained from a vector image by projecting the 
color vector for each pixel of said image onto the color 
vector of said hue. 

38. The apparatus of any of claims 8 through 15 and 
22, further comprising means for three-dimensional surface 
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reconstruction from multiview or multi-illumination or 
single view, single illumination image or images. 

39. The apparatus of any of claims 8 through IS, 
further comprising means for three-dimensional surface 
reconstruction from an image by a shape-from shading method. 

40. The apparatus of any of claims 8 through 15, 
further comprising means for reconstructing subsurface 
layers by employing a combination of images, each said image 
representing the energy of a specified spectral band 
reflected from said surface and said subsurface, each said 
image representing structure to the depth of energy 
penetration for said spectral band. 

41. The apparatus of claim 40 wherein each said 
image representing the energy of a specified spectral band 
is obtained by spectral projection. 

• 42. The apparatus of any of claims 8 through 15 
further comprising means for segmenting a* image of a body 
surface into regions of .interest and noninterest. 

43. The apparatus of claim 42 wherein said 
segmenting means comprises means for segmenting regions of 
lesion from regions of background. 

44. The apparatus of claim 42 wherein said 
segmenting means comprises means for segmenting regions of 
specular: reflection from regions of interest. 

45. The apparatus of claim 42 wherein said 
segmenting means comprises means for using local bimodal 
histogram segmentation to segment said image. 

46. The apparatus of claim 45 wherein said means for 
usmg local bimodal histogram segmentation to segment said 
image comprises means for scoring bimodal histograms to 
quantify edge strength. 

47. The apparatus of claim 42, further comprising 
means for performing hair elimination prior to segmentation 
by said segmentation means. 

48. The apparatus of claim 47 wherein said means for 
performing, ha;Lr elimination comprises means for. using an 
appropriate gray-scale morphological operation. 
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45. The apparatus of claim 42, further comprising 
means for performing a morphologic opening and closing 
operation on said image prior to segmentation by said 
segmentation means. 

50. The apparatus of any of claims 8 through IS 
wherein said quantitative feature extraction means comprises 
means for counting occurrences of a feature in an image. 

51. The apparatus of claim 50 wherein said feature 
is a lesion. 

52. The apparatus of any of claims 8 through 15 
wherein said quantitative feature extraction means comprises 
means for quantifying morphological and topological features 
of a three-dimensional representation of a body surface. 

53. The apparatus of any of claims 8 through 15 
wherein said quantitative feature extraction means comprises 
means for quantifying the asymmetry of a lesion in an image. 

54. The apparatus of claim 53 wherein said asymmetry 
quantifyng means comprises means for quantifying image 
border asymmetry by determining the roundness of the lesion. " 

55. The apparatus of claim 53 wherein said asymmetry 
quantifying means comprises means for quantifying image 
border asymmetry by determining the eccentricity of the 

lesion . 



56. The apparatus of any of claims 8 through 15 
wherein said quantitative feature extraction -means comprises 
means for quantifying border irregularity of a lesion. 

57. The apparatus of claim 56 wherein said border 
irregularity quantifying means employs statistical 
properties of the distribution as a function of angle of the 
distance between the centroid of said lesion to said border 
as a quantification of the border irregularity. 

58. The apparatus of claim 56 wherein said border 
irregularity quantifying means employs the convex deficiency 
of the border as a quantification of the border 
irregularity. 

59. The apparatus of claim 56 wherein said border 
irregularity quantifying means employs the convex excess of 
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j the border as a quantification of the border irregularity. 

I 60. The apparatus of claim 56 wherein said border 

| irregularity quantifying means employs means for determining 

a measure of the compactness of the lesion as a 
j quantification of the border irregularity. 

| 61. The apparatus of claim 56 wherein said border 

means employs means for determining 
j or estimating the fractal dimension of the border as a 

j quantification of the border irregularity. 

62. The apparatus of claim 56 wherein said border 

irregularity quantifying means employs means for determining 
j the distinctness of the border as a quantification of the 

border irregularity. 
! 63. The apparatus of claim 19 or 62 wherein said 

border distinctness determining means scores the bimodal 

histograms during segmentation as a quantification of the 

border irregularity. 

64. The apparatus of claim 56 wherein said border 
irregularity quantifying means comprises means for analyzing 
parameters derived from spectral representations of the 
border . 

j 65. The apparatus of any of claims 8 through IS 

. wherein said quantitative feature extraction means comprises 

; means for quantifying the color distribution in an image or 

region of an image.. ■ 

66. The apparatus of claim 20 or 65 wherein said 
color distribution quantifying means obtains quantitative 
statistical parameters from said color distribution. 

67. The apparatus of claim 20 or 65, further 
comprising means for displaying said color distribution as a 
set of points on a color wheel which displays all possible 
colors of a given color space model at a given level of 
intensity. 

68. The apparatus of claim 65, further comprising 
means for displaying said color distribution as histograms 
of hue and saturation. ... 
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69. The apparatus of any of claims 8 through 15 
wherein said quantitative feature extraction means comprises 
means for quantifying the maximum and minimum diameters and 
the area of a lesion. 

70. The apparatus of any of claims 6 through 15 
wherein said quantitative feature extraction means comprises 
means for quantifying statistical features of the surface 
elevation of a body surface. 

71.. The apparatus of any of claims 8 through 15 
wherein said quantitative feature extraction means comprises 
means for computing Nth order moment distances between 
scalar images . 

72. The apparatus of any of claims 8 through 15 and 
23 wherein said quantitative feature extraction means 
comprises means for characterizing the texture of any scalar 
or vector image or image region. 

73. The apparatus of claim 72 wherein said texture 
characterizing means determines the fractal dimension of a 
scalar image as a characterization of the texture. 

74. The apparatus of claim 72 wherein said texture 
characterizing means computes parameters from the spatial 
gray level dependence matrix as a characterization of the 



texture . 



75. The apparatus of claim 72 wherein said texture 
characterizing means computes parameters from the run length 
gray level matrix. as a characterization of the texture. 

76. The apparatus of claim 74, further comprising 
means for determining the local variance of said parameters 
as a further characterization of the texture. 

77. The apparatus of claim 75, further comprising 
means for determining the local "variance" of said parameters 
as a further characterization of the texture. 

78. The apparatus of claim 72 wherein said texture 
characterizing means analyzes the extrema density as a 
characterization of the texture. 
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79. The apparatus of claim 72 wherein said texture 
characterizing means analyzes the Fourier domain fractal 
dimension as a characterization of the texture. 

80. The apparatus of claim 72 wherein said texture 
characterizing means analyzes the power spectrum as a 
characterization of the texture. 

81. The apparatus of claim 72 wherein said texture 
characterizing means analyzes Nth order probability density 
function signatures as a characterization of the texture. 

82. The apparatus of claim 72 wherein said texture 
characterizing means characterizes the texture in a region 
of interest determined from morphologic operations on a 
binary image. 

83. The apparatus of any of claims 8 through 15 
-herein said quantitative feature extraction means comprises 
means for computing statistical parameters of multispectral 
distributions . 

84. The apparatus of claim 26 wherein said visual 
display comprises 

means for displaying at least one of a plurality of 
stored images, 

means for displaying one or more arrays, each said 
array comprising a plurality of image icons corresponding to 
some or all of said plurality of stored images, and 

operator control means for selecting which of said 
stored^images is represented in said arrays of image icons 
and which of said stored images is displayed. 

85. The apparatus of claim 84 wherein said operator 
control means comprises means for using said image icons to 
indicate which of said images represented by said image 
icons is to be displayed, processed, or analyzed according 
to a given command. 

8S. The apparatus of claim 85, further comprising 
means for coding each displayed image and image icon to 
indicate relationships between said displayed images and 
-.age icons according to a coding scheme employing graphics 
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87. The apparatus of claim 86 wherein said coding 
scheme comprises a colored border or box for each high 
magnification displayed image and image icon and 
corresponding colored boxes superimposed over each lower 
magnification displayed image and image icon, whereby the 
location of each high magnification image in each lower 
magnification image is indicated. 

88. The apparatus of claim 26 wherein said visual 
display comprises means for displaying profiles of any one- 
dimensional function. 

89. The apparatus of claim 26 wherein said visual 
display comprises means for displaying pseudo three- • 
dimensional diagrams of any two-dimensional function or 
image . 

90. The apparatus of claim 26 wherein said visual 
display comprises means for displaying pseudo three- 
dimensional diagrams of three-dimensional surface 
reconstructions . 

91. The apparatus of claim 26 wherein said visual 
display comprises means for displaying subsurface 
reconstructions . 

92. The apparatus of claim 26 wherein said visual 
display comprises means for displaying selected attributes 
of an image . 

93. The method of claim 2, further comprising 
extracting a selected attribute or representation from said 
digital image. 

94. The method of claim 93, wherein the region of 
skin includes a pigmented lesion, and said extracting step 
comprises extracting the rets pigment or capillary patterns 
from said digital- image. 1 

95. The method of claim 2, further comprising 
quantitatively extracting a selected feature from said 
digital image. 

96. The method of claim 2, 93, or 95, comprising 
placing a transparent member over said moistener on the 
skin . 
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97. The method of claim 96 wherein said transparent 
member comprises a transparent flexible membrane. 

98. The method of claim 3, further comprising 
applying a moistener onto the skin region. 

»9. The method of claim 2 or 3 wherein said 
moistener comprises oil. 

100. The method of claim 99 wherein said moistener 
comprises mineral oil . 

101. The method of claim 3 wherein said polarizers 
are plane polarizers or circular polarizers. 

102,. The method of claim 101 wherein one of said 
polarizers can be rotated about the illumination path with 
respect to the other. 

103. The method of claim 101 or 102 wherein the 
principle transmission axes of said polarizers are 
perpendicular to one another. 

104. The method of claim 101 or 102 wherein the 
principle transmission axes of said polarizers are parallel 
to one another. 

105. The method of claim 4, wherein said first and 
subsequent multispectral digital images are higher 
magnification images, said method further comprising 

recording at -about said first time a first lower 
magnification digital Image that includes a portion of the 
surface that includes the region and recording at about said 
subsequent time a subsequent lower magnification digital 
image that includes a portion of the surface that includes 
the region. 

106. The methbd of claim 5 wherein said clinically 
accepted diagnostic tool comprises histopathologic analysis 
of a sample of the lesion. 
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